perf[array]: outline the array parts data from the DynArray#7750
perf[array]: outline the array parts data from the DynArray#7750robert3005 merged 14 commits intodevelopfrom
Conversation
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> # Conflicts: # vortex-array/src/array/erased.rs
Benchmarks: Random AccessVortex (geomean): 0.977x ➖ unknown / unknown (1.006x ➖, 2↑ 0↓)
|
Benchmarks: CompressionVortex (geomean): 1.017x ➖ unknown / unknown (1.039x ➖, 2↑ 21↓)
|
Benchmarks: TPC-H SF=10 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (1.002x ➖, 0↑ 1↓)
datafusion / vortex-compact (1.044x ➖, 0↑ 0↓)
datafusion / parquet (0.984x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.954x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.979x ➖, 0↑ 0↓)
duckdb / parquet (0.966x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=1 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.884x ➖, 2↑ 1↓)
datafusion / vortex-compact (1.013x ➖, 1↑ 1↓)
datafusion / parquet (1.014x ➖, 0↑ 3↓)
duckdb / vortex-file-compressed (1.006x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.036x ➖, 0↑ 0↓)
duckdb / parquet (0.998x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: TPC-DS SF=1 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.987x ➖, 1↑ 1↓)
datafusion / vortex-compact (0.986x ➖, 2↑ 1↓)
datafusion / parquet (0.995x ➖, 2↑ 0↓)
duckdb / vortex-file-compressed (0.990x ➖, 0↑ 2↓)
duckdb / vortex-compact (0.995x ➖, 2↑ 0↓)
duckdb / parquet (1.002x ➖, 1↑ 3↓)
duckdb / duckdb (0.999x ➖, 1↑ 3↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=1 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.997x ➖, 0↑ 0↓)
datafusion / vortex-compact (0.998x ➖, 0↑ 0↓)
datafusion / parquet (0.971x ➖, 1↑ 1↓)
datafusion / arrow (0.965x ➖, 3↑ 0↓)
duckdb / vortex-file-compressed (1.003x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.008x ➖, 0↑ 0↓)
duckdb / parquet (1.007x ➖, 0↑ 1↓)
duckdb / duckdb (1.021x ➖, 0↑ 2↓)
Full attributed analysis
|
Benchmarks: Statistical and Population GeneticsVerdict: No clear signal (low confidence) duckdb / vortex-file-compressed (0.907x ➖, 2↑ 0↓)
duckdb / vortex-compact (0.931x ➖, 1↑ 0↓)
duckdb / parquet (0.939x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=10 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.979x ➖, 0↑ 0↓)
datafusion / vortex-compact (0.986x ➖, 0↑ 0↓)
datafusion / parquet (0.985x ➖, 0↑ 0↓)
datafusion / arrow (0.990x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.991x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.992x ➖, 0↑ 0↓)
duckdb / parquet (0.995x ➖, 0↑ 0↓)
duckdb / duckdb (0.996x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: FineWeb S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.983x ➖, 0↑ 0↓)
datafusion / vortex-compact (0.967x ➖, 1↑ 1↓)
datafusion / parquet (1.015x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed (1.080x ➖, 0↑ 0↓)
duckdb / vortex-compact (1.007x ➖, 0↑ 0↓)
duckdb / parquet (0.999x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: Clickbench on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (1.002x ➖, 0↑ 1↓)
datafusion / parquet (1.007x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.973x ➖, 7↑ 1↓)
duckdb / parquet (1.005x ➖, 0↑ 1↓)
duckdb / duckdb (0.984x ➖, 3↑ 0↓)
Full attributed analysis
|
Benchmarks: PolarSignals ProfilingVortex (geomean): 1.015x ➖ datafusion / vortex-file-compressed (1.015x ➖, 0↑ 0↓)
|
Benchmarks: FineWeb NVMeVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.999x ➖, 1↑ 0↓)
datafusion / vortex-compact (0.998x ➖, 0↑ 1↓)
datafusion / parquet (1.007x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.959x ➖, 1↑ 0↓)
duckdb / vortex-compact (1.007x ➖, 0↑ 0↓)
duckdb / parquet (1.001x ➖, 0↑ 0↓)
Full attributed analysis
|
Merging this PR will degrade performance by 16.36%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | filter_all_true[100000] |
7.7 µs | 8.8 µs | -12.59% |
| ⚡ | Simulation | bench_many_codes_few_values[1024] |
436.9 µs | 396.3 µs | +10.24% |
| ⚡ | Simulation | encode_varbinview[(1000, 2)] |
244.3 µs | 199.5 µs | +22.44% |
| ❌ | Simulation | filter_all_true[250000] |
7.7 µs | 8.8 µs | -12.59% |
| ⚡ | Simulation | execute_scalar_struct_simple |
516.5 µs | 468.2 µs | +10.31% |
| ⚡ | Simulation | dict_canonicalize_zipfian[16, 1000] |
89.3 µs | 50 µs | +78.52% |
| ⚡ | Simulation | new_bp_prim_test_between[i16, 32768] |
134.7 µs | 120.3 µs | +11.92% |
| ⚡ | Simulation | new_bp_prim_test_between[i32, 16384] |
108.8 µs | 93.8 µs | +15.98% |
| ❌ | Simulation | patched_take_10k_adversarial |
229.5 µs | 259.1 µs | -11.43% |
| ⚡ | Simulation | new_bp_prim_test_between[i32, 32768] |
169.2 µs | 140.2 µs | +20.72% |
| ⚡ | Simulation | new_bp_prim_test_between[i64, 16384] |
143.9 µs | 114.4 µs | +25.76% |
| ⚡ | Simulation | patched_take_10k_contiguous_patches |
288.2 µs | 258.9 µs | +11.31% |
| ⚡ | Simulation | patched_take_10k_random |
300.5 µs | 271.2 µs | +10.8% |
| ❌ | Simulation | take_10k_dispersed |
240.7 µs | 284.5 µs | -15.39% |
| ❌ | Simulation | take_10k_first_chunk_only |
226.8 µs | 271.2 µs | -16.36% |
| ⚡ | Simulation | decompress_rd[f32, (100000, 0.1)] |
583.9 µs | 495.9 µs | +17.75% |
| ⚡ | Simulation | decompress_rd[f32, (100000, 0.01)] |
583.9 µs | 495.8 µs | +17.76% |
| ❌ | Simulation | decompress_rd[f32, (100000, 0.0)] |
496.9 µs | 584.2 µs | -14.94% |
| ❌ | Simulation | decompress_rd[f64, (10000, 0.1)] |
123.4 µs | 139.2 µs | -11.38% |
| ⚡ | Simulation | decompress_rd[f64, (100000, 0.1)] |
1,022 µs | 843.2 µs | +21.21% |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Comparing ji/vtable-array-outer (ee1a899) with develop (fb0f605)
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> # Conflicts: # vortex-array/src/array/erased.rs # vortex-array/src/array/mod.rs
|
@joseph-isaacs I made it compile but I maybe there's a better way |
|
This seems like a reasonable why to do this for now |
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
File Sizes: PolarSignals ProfilingNo file size changes detected. |
File Sizes: FineWeb NVMeNo file size changes detected. |
File Sizes: TPC-H SF=1 on NVMENo file size changes detected. |
File Sizes: TPC-DS SF=1 on NVMENo file size changes detected. |
File Sizes: TPC-H SF=10 on NVMENo file size changes detected. |
File Sizes: Statistical and Population GeneticsNo file size changes detected. |
File Sizes: Clickbench on NVMEFile Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
Totals:
|
Change the internals of
ArrayRefto allow access to common fields without dyn dispatch.Applied the following renames:
ArrayInner->ArrayData<V as VTable>::ArrayData-><V as VTable>::TypedArrayDataDynArray->DynArrayData